Antibiotics Time Machine is NP-hard

The antibiotics time machine is an optimization question posed by Mira et al. on the design of antibiotic treatment plans to minimize antibiotic resistance. The problem is a variation of the Markov decision process. These authors asked if the problem can be solved efficiently. In this paper, we show that this problem is NP-hard in general. 1. Problem statement and results Resistance to antibiotics in bacteria is a central problem in medicine. One method for mitigating its consequences is to rotate the use of antibiotics. A natural question arises: what is the optimal sequence of antibiotics one should apply? In this work, we follow the model proposed by Mira et al. [MCG14]. Suppose we have a population of bacteria, all of the same unmutated genotype (called the wild type). We are given a set of antibiotics. For each bacteria of a given type, an antibiotic mutates it to another type with some known probability. The problem is to find a sequence of antibiotics (called a treatment plan) that maximizes the fraction of bacteria returning to the wild type. This is equivalent to minimizing the fraction of antibiotics-induced mutations in the bacterial population. Here is a concrete description of the model. Let S be a set of d states, where each state is a bacterial genotype. A d×d matrix T = (tij) is a transition matrix if tij ∈ [0, 1] and each row sums to 1. Let T be a finite set consisting of K transition matrices, each corresponding to the effect of a given antibiotic on the genotypes of the bacterial population. That is, tij is the probability that a bacteria of genotype i will have genotype j after being treated with the corresponding antibiotic. The standard (d− 1)-dimensional simplex ∆d−1 is given by ∆d−1 = {(x1, x2, . . . , xd) ∈ R | ∑d i=1 xi = 1 and xi ≥ 0 for all i}. Let s = (1, 0, . . . , 0) ∈ ∆d−1 be the starting state of the population, where all bacteria are of the first state (the wild type). Let N ∈ N be the length of the treatment plan. The time machine TM(d,K,N) is the solution to the following optimization problem maximize sT1T2 · · ·TNs subject to T1, . . . , TN ∈ T . (1) The term time machine, coined by Mira et al., alludes to the idea of reversal to the wild type of unnecessary mutations done by antibiotic treatments. They proposed and solved a specific numerical instance of the time machine for 16 genotypes of the TEM β-lactamase, with a specific model of mutation, and a set of up to six antibiotics. Their algorithm is to enumerate all possible treatment plans of a given length with the given set of antibiotics and output the optimal one. With K antibiotics, there are K many treatment plans of length N , and thus direct enumeration is not an efficient algorithm. Mira et al. raised the question of whether an efficient algorithm exists. In this paper, we show that the general problem is NP-hard. Tran–Yang 2 May 12, 2015 Theorem 1. The time machine optimization problem in (1) is NP-hard. We first consider an easier approximation problem: for a given threshold α ∈ [0, 1], decide whether there exists a treatment plan of length N such that sT1T2 · · ·TNs ≥ α subject to T1, . . . , TN ∈ T . (2) Clearly if one can solve (1) in polynomial time, then one can also solve (2) in polynomial time. Our main result states that the latter problem is NP-hard, and thus Theorem 1 follows as a corollary. Theorem 2. The time machine decision problem in (2) is NP-hard. Our proof relies on reducing 3-SAT to a special instance of (2) where α = 1. Since 3-SAT is NP-hard [Kar72], we have that (2) is also. The reduction construction is described in Section 2. The proof that this is a correct reduction is in Section 3. 1.1. Related literature. The time machine may also arise in large scale robotics control, where one has a large number of robots with d possible states, and each Ti is an instruction which changes each robot’s state independently at random. In general, it can be viewed as a Markov decision problem with infinitely many states. Indeed, if there was only one bacteria instead of a population, one may apply an antibiotic Ti, check the new state of the bacteria, and repeat. In this case, the time machine is a Markov decision process on d states, which can be solved by dynamic programming. When there is a population of bacteria, the population state is a point in the standard (d − 1)-dimensional simplex ∆d−1, which is an infinite set. There is a large literature on Markov decision processes, see [FSA02] and references therein. However, we are unaware of any existing results on the time machine, or the equivalent of Theorem 1. Closely related to the time machine is the partially observable Markov decision process [DS13], where the underlying system is assumed to be in one of the unobservable d states, and s ∈ ∆ reflects the uncertainty over the true state. It would be interesting to see if techniques from this setup can yield polynomial time approximations to the time machine. 1.2. Open problems. Many interesting questions remain on the antibiotics time machine. We mention two examples. The last question was raised by Joe Kileel. (i) For what classes of antibiotics T is the problem solvable in polynomial time? Are there biologically relevant instances? (ii) When does the limit limN→∞maxTi∈T sT1T2 · · ·TNs exist? If so, how fast is the convergence? In this case, for largeN , can one produce an approximate time machine in polynomial time? 2. Reduction construction Let X = {x1, . . . , xn} be a set of Boolean variables taking Boolean values {+,−}. A triple (εixi, εjxj, εkxk) with εi, εj, εk ∈ {+,−} is called a clause. Let C = {c1, . . . , cm} be a set of clauses. Write (xi = vi)i to mean xi = vi for i = 1, . . . , n. An assignment (xi = vi)i satisfies a clause c = (εixi, εjxj, εkxk) if (εi, εj, εk) 6= (vi, vj, vk). Without loss of generality, assume each x ∈ X actually occurs (negated or not) somewhere. The 3-SAT problem is: given a set of variables and clauses, decide whether there exists an assignment that satisfies all clauses. In this section, we reduce a 3-SAT instance to an instance of (2). Antibiotics Time Machine is NP-hard Tran–Yang 3 Let N = m + 2 and α = 1. Create d = 3n + m + 3 states and K = 7m + 2 transition matrices. Create the start state s, a “death” state d, and a “tally” state f . For each variable x ∈ X, create states x,x−,x+. For each clause c = (εixi, εjxj, εkxk) ∈ C, create state c. We identify each state with its characteristic (row) vector and define each transition matrix by its action on these basis vectors and extend linearly. For each choice of (vi, vj, vk) ∈ {+,−} r {(εi, εj, εk)}, create a transition matrix T = T vi,vj ,vk c as follows: zT =  d z = s f z = c x` ` z = x`, ` ∈ {i, j, k} d z = x` ` , ` ∈ {i, j, k} z otherwise. Define a starting matrix S by zS = { p (∑ x∈X x + ∑ c∈C c ) z = s d otherwise, where p = 1/(n+m). Lastly, define a final matrix F by zF =  s z ∈ {xi i : vi ∈ {+,−}} s f